Introducing LiteRT Next: A new set of APIs that improves and simplifies on-device hardware acceleration.

Get Started with LiteRT Next

The LiteRT Next APIs are not compatible with the LiteRT APIs, so applications using LiteRT must completely migrate to LiteRT Next in order to make use of the features and capabilities provided by the new APIs. Applications cannot use the TF Lite Interpreter APIs and Compiled Model APIs interchangeably.

LiteRT Next provides APIs for Kotlin and C++. Applications using a LiteRT SDK in other languages should continue using LiteRT.

Android dependencies

To migrate an Android application using LiteRT, replace the dependency from com.google.ai.edge.litert to com.google.ai.edge.litert:litert:2.0.0-alpha.

With LiteRT, the GPU accelerator is available as a delegate in a separate library (com.google.ai.edge.litert:litert-gpu). With LiteRT Next, the GPU accelerator is included in the LiteRT Next package. For more information, see GPU with LiteRT Next.

You can add the LiteRT Next package to your build.gradle dependencies:

dependencies {
  ...
  implementation `com.google.ai.edge.litert:litert:2.0.0-alpha`
}

Code changes

Applications using LiteRT will have to substitute code that uses the TFLite Interpreter API for the code using the Compiled Model API. The following shows the major changes required to migrate to LiteRT Next. For more details, see the LiteRT Next API reference.

Code changes in C++

To migrate an application using C++, substitute the following key snippets:

	LiteRT (TFLite Interpreter)	LiteRT Next (`CompiledModel`)
Load a Model	`FlatBufferModel::BuildFromFile() InterpreterBuilder(...)`	`Model::CreateFromFile("mymodel.tflite")` Note: No separate builder step
Initialize Runtime	`builder(&interpreter), interpreter->AllocateTensors()`	`CompiledModel::Create(env, model, kLiteRtHwAcceleratorCpu)` Note: No manual memory allocation step
Use Accelerators	`interpreter->ModifyGraphWithDelegate(...)`	`CompiledModel::Create(env, model, kLiteRtHwAcceleratorGpu)`
Run a Model	`interpreter->Invoke()`	`compiled_model->Run(inputs, outputs)`

Code changes in Kotlin

To migrate an application using Kotlin, follow the following key steps:

Set up model and runtime

With LiteRT, you load a model, set up acceleration, and initialize the runtime in different steps:

// Load the model
val modelBuffer: MappedByteBuffer =
  FileUtil.loadMappedFile(appContext, "model.tflite")

// Initialize runtime
val options = Interpreter.Options()
val interpreter = Interpreter(modelBuffer, options)
interpreter.allocateTensors()

// Use accelerators
aval gpuDelegate = GpuDelegate()
options.addDelegate(gpuDelegate)

With LiteRT Next, you load the model, specify the acceleration, and initialize the runtime at the same time:

val model =
CompiledModel.create(
  context.assets,
  "model.tflite",
  CompiledModel.Options(Accelerator.GPU)
)

Run inference

To run the model with LiteRT:

val input = FloatBuffer.allocate(data_size)
val output = FloatBuffer.allocate(data_size)
interpreter.run(input, output)

To run the model with LiteRT Next:

val inputBuffers = model.createInputBuffers()
val outputBuffers = model.createOutputBuffers()
model.run(inputBuffers, outputBuffers)

Other libraries

The LiteRT Next APIs are only available in Kotlin and C++. Applications using the LiteRT SDKs in other languages cannot migrate to LiteRT Next.

Applications using the LiteRT in the Play Services runtime cannot migrate to LiteRT Next, and should continue using the play-services-tflite runtime. The Task Library and Model Maker libraries cannot migrate to LiteRT Next, and should continue using the TensorFlow Lite APIs.